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Abstract 

This study investigates the validity of equivalency theory among 63 students by comparing two 
introductory upper-division human resource management courses: one taught online, the other in 
a traditional classroom. Commonalities included same term, same professor, and identical 
assignments/tests in the same order, thus allowing a direct comparison of course outcomes. 
MANCOVA results supported equivalency theory, and further suggest that the online learning 
pedagogy may be superior in its overall effect on student performance. 
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Introduction 

While there is emerging research suggesting that courses delivered online produce at least 
comparable learning outcomes relative to traditional classroom-based courses when using similar 
instructional methods (Sitzmann, Kraiger, Stewart, and Wisher, 2006), the extent to which this 
conclusion can be reached based upon evidence varies substantially within the management 
discipline. While subjects such as management science (Dellana, Collins, and West, 2000; 
McLaren, 2004), management information systems (Clouse and Evans, 2003; Piccoli, Ahmad, 
and Ives, 2001; Webb, Gill, and Poe, 2005), organizational behavior (Friday, Friday-Stroud, 
Green, and Hill, 2006; Meisel and Marx, 1999), and strategy (Arbaugh, 2000; Friday et ah, 2006) 
have been subjects of “online vs. classroom” comparison studies, courses in the human resources 
discipline have yet to be extensively considered in this body of research. 

Another concern regarding comparison studies of online and classroom-based learning is the 
focus of the analysis. While some of these studies consider student characteristics such as gender 
(Arbaugh, 2000; Friday et ah, 2006) or learning style (Clouse and Evans, 2003) in their designs, 
in measuring course design characteristics they tend to focus almost exclusively on exam 
performance. Such a perspective is particularly problematic for the management discipline since 
these courses often use a variety of approaches in addition to exams to assess student learning 
(Bailey, Sass, Swiercz, Seal, and Kayes, 2005; Fornaciari and Dean, 2005; Williams, Duray, and 
Eddy, 2006). This suggests that the design of online courses within the management discipline 
should be considered more comprehensively in comparison studies. Simonson, Schlosser and 
Hanson (1999) offered an "equivalency theory," stating that courses should provide equivalent 
learning experiences for all students, regardless of the method of delivery (traditional classroom, 
interactive telecommunication systems, or online). For this study, two sections of an 
undergraduate course were offered during the same quarter by one professor using identical 
syllabi and assessment instruments. The two courses differed only in the presentation format: one 
was a traditional classroom with limited online exercises and one was entirely online. These 
increased experimental controls allow for a more rigorous testing of equivalency theory than 
previous studies have provided. 
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Equivalency Theory and Online Learning 

Simonson and colleagues (1999) developed equivalency theory as a means to integrate previous 
theories of distance education into a uniquely American perspective in light of recent advances in 
telecommunications technologies. The theory is intended to insure that distance education does 
not become an inferior form of education, and in fact may not even be a distinct field of 
education. Equivalency theory argues that the more equivalent the learning experiences of 
distance learners to that of local learners, the more equivalent the educational outcomes for all 
learners. Such an approach suggests that course designers create learning experiences of 
equivalent value for learners regardless of the course delivery medium, allowing that the 
experiences themselves could be different. 

Recent discussions of equivalency theory have focused on how one establishes “equivalence.” 
Watkins and Schlosser (2000) argued that equivalence should be determined based upon 
demonstrated learner accomplishment rather than instructional time-based criteria. Such an 
approach suggests the need to evaluate learner performance on similar types of assessments 
using a broader range of assessments than final exam scores, which tends to be the measure of 
choice in many “online vs. classroom” studies (Weber and Lennon, 2007). While recent 
comparison studies in business education have begun to compare other outcomes such as 
participation patterns, class projects, and overall course grade (Arbaugh, 2000; Friday et ah, 

2006; Weber and Lennon, 2007), the range of activities considered in such studies to date 
remains relatively limited. Therefore, it is the intent of this study is to examine the question 
whether online teaching is less, as, or more effective than traditional classroom teaching across a 
variety of assessment methods. To this end, a methodology to test equivalency theory is 
developed, and results are discussed in terms of their potential support of equivalency theory. 
Implications of the findings toward the further pursuit of equivalency theory relevance and 
usefulness are considered. 


Methodology 

Research testing the support of equivalency theory needs to move in the direction of increased 
experimental controls using appropriate subjects. Interpreting the equivalency guidelines of 
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Simonson and colleagues (1999), we incorporate the following controls on undergraduate 
subjects: two equivalent courses were administered at the same time by the same professor, with 
virtually the only difference being different pedagogy (one course was conducted entirely online, 
the other course was held in a traditional classroom). Thus, by presenting the same course 
material in the same order, with identical assignments, quiz questions, and final reports, student 
outcomes in the traditional course can be more rigorously compared to the learning experienced 
by the online students. The course used in this study was “Management of Human Resources,” 
an introductory class designed to familiarize students with various aspects of the HR discipline. 
This junior-level course covers the basics of human resource management including definitions, 
current practices, and a review of HR laws. 

The online course used in this study was developed to meet students' demands for a course with 
greater time flexibility at a branch campus approximately 100 miles from the main campus of a 
university located in the northwestern United States. Most of the online students were transfer 
students with an associate's degree from a community college where many of the courses are 
offered online. The professor who administered both the traditional and online courses had 
taught both types of courses numerous times, however, this was the first time the professor 
taught two sections of the same course, during the same quarter, using different delivery 
methods. 

Prior to accepting the task of teaching the web-based course, the professor had chosen to modify 
the traditional course to accommodate a different textbook with more web resources for students 
and more exercises and problems at the end of each chapter. Therefore, a new online-savvy 
textbook was used for both the online and traditional classroom courses. The syllabi for the 
courses were essentially identical, except that the syllabus for the online course contained 
information on how to access the course management system. Blackboards'^, and listed deadlines 
for completing each online quiz or problem. Both syllabi stated, for instance, that Chapters 1 and 
2 would be covered in Week 1. While the traditional classroom students had to attend classes and 
participate in Chapter 1 and 2 discussions on Tuesday and Thursday, the online students were 
required to participate in an online asynchronous discussion involving Chapter 1 and 2 materials, 
responding no later than Sunday midnight. Three different types of instruments were used to 
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assess student learning in both sections of the junior- level, introductory human resource 
management course: chapter quizzes, online discussions, and written reports. 

The traditional classroom pedagogy consisted of lectures and in-class discussions, with 
homework from the end-of-chapter exercises and problems. The online classroom pedagogy 
differed in that online students didn't hear lectures and in lieu of homework were required to 
participate in online threaded discussions centered on the same end-of-chapter exercises and 
problems as the classroom students. Students in both courses had access to Blackboards'^ and 
were encouraged to take online practice quizzes (in Blackboards'^) supplied by the textbook 
publisher. Practice quizzes were in the same multiple choice format as the graded quizzes for 
both classes. To familiarize the traditional classroom students with the Blackboards'^ system, 
they were required to attend a seminar in a computer lab where they were guided through the 
software and then required to participate in a trial online threaded discussion. In contrast, the 
online students were required to have some familiarity with computers prior to registering for the 
class; therefore, no such lab training was offered them. 

The 37 students enrolled in the classroom section of the course were mostly traditional, 
residential undergraduate college students at the main campus. The 26 students in the online 
course were geographically distant so never met the professor face-to-face; contact was limited 
to e-mail, online discussions, and occasional phone calls from students to the professor. These 
students were mostly non-traditional, with many of them working full-time and taking the course 
due to its time flexibility (as self-reported in their introductory online discussions). The course 
initially contained 28 students but one dropped due to personal time constraints, while one other 
dropped the course due to lack of computer skills. 

The students from the two courses can be compared on a number of demographic variables to 
determine how the two samples differed. For each individual the variables included grade point 
average, credits earned to date, and age. All variables were measured at the start of the class. 
Other variables included number of credits taken during the same quarter as the test class, sex, 
and whether the student was a business major or not. Results from either t-tests or chi-square 
tests, depending on the level of data contained in the variable being tested, are shown in Tables 1 
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and 2. All data for these tests were collected from university student records. Based on the tests, 
a number of significant demographic differences exist between the two classes. The students in 
the online course, in general, had higher grade point averages, were older, took fewer credits 
during the same term, and were more likely to be business majors. There were no significant 
differences in the students from the two classes in terms of credits earned to date and sex. 


TABLE 1: Comparison of Ratio-level Demographic Variables by Class 


Variable 

Class Type 

N 

Mean 

Std. Dev. 

t 

Sig. 

GPA when class started 

Traditional 

34 

2.67 

.55 




Online 

24 

3.06 

.48 

-2.823 

.007 

Credits earned to date 

Traditional 

37 

131.16 

28.51 




Online 

26 

137.81 

23.02 

-.984 

.329 

Age in Years 

Traditional 

37 

22.62 

1.72 




Online 

26 

28.88 

7.11 

-4.404 

.000 

Number of credits taken 

Traditional 

37 

15.05 

1.96 



during term of class 

Online 

26 

12.42 

3.95 

-3.136 

.004 


TABLE 2: Comparison of Nominal-level Demographic Variables by Class 


Class Type 

Variable Counts and Percentages 

Chi- 

Square 

d.f. 

Sig. 


Business Major 

Other Major 




Traditional 

9 

28 





24.3% 

75.7% 

17.01 

1 

.000 

Online 

20 

6 





76.9% 

23.1% 





Female 

Male 




Traditional 

16 

21 





43.2% 

56.8% 

.688 

1 

.407 

Online 

14 

12 





53.8% 

46.2% 
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All students in both versions of the course were required to complete 16 quizzes, each consisting 
of 15 multiple-choice questions, one per textbook chapter; the quiz questions for each chapter 
were identical and presented in the same order for both courses. The traditional students were 
given 14 of the quizzes in class via traditional paper and pencil methods; they were required to 
complete two of the quizzes online on a computer of their choice, outside of the classroom, on 
their own time. These two online quizzes were required so traditional classroom students would 
receive some exposure to the online learning environment, a format they might encounter in 
future courses. Their first online quiz (Chapter 16, Global Issues) followed a typical lecture on 
the associated course material, and students had access to on-campus computer labs immediately 
following the lecture, although most preferred to take the quiz later. For the second quiz (Chapter 
11, Benefits), the students were asked to read the chapter and complete the quiz on their own. 

The professor was available during normal office hours to answer any questions these traditional 
classroom students might have, but no students sought assistance with the subject matter. 

For the online course students, quizzes for each chapter were available on a one-time basis with a 
25-minute time limit (same timeframe allowed the traditional students), and each 15-question 
quiz was to be completed during the week shown in the syllabus. Access to the quiz was denied 
after the one-week timeframe that corresponded to the traditional classroom's week. The online 
students had access to the professor via e-mail and phone, but few chose to use these methods. 
This study compares the results of the two classes' online quizzes since these were identical 
experiences for both groups, except the traditional classroom students had a typical lecture prior 
to the first online quiz and were given limited guidance prior to the second online quiz. 

Online threaded discussions provided a means for the online students to exchange ideas, taking 
the place of the in-class discussions and homework. Students were to submit a response to an 
end-of-chapter exercise or question that could be read and commented on by other class 
participants in asynchronous time. A rubric was used to rate the responses, with points associated 
with various types of responses: low points were given to mere opinions, higher points were 
given when students related their answers to the textbook material, and the highest points 
rewarded students who not only related responses to the text but also incorporated additional 
information from websites or from speaking with professionals. The rubric was attached to the 
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syllabus and readily available to students in both the online course and the traditional classroom 
course. While the online course students were required to respond to thirteen online threaded 
discussion questions, the traditional classroom students were only asked to respond to three; the 
other questions and exercises were part of typical classroom discussions. Both classes responded 
to identical threaded discussion problems from chapters 5, 7 and 11, and both classes had the 
same time restrictions placed on them for responses. Grades for these three discussion questions 
are compared since they are common elements for both classes, although students would have 
acquired the information differently (in-class discussion versus self-learning). 

As the final assessment piece for the course, each student in each course section was required to 
submit a written report, due the last week of class. Students in both sections of the course were 
given the option of e-mailing the reports to the professor or handing in a hard copy. All reports 
were graded from hard copies; those e-mailed to the professor were printed out prior to grading. 
The same grading rubric attached to both syllabi was used for all grading. Report requirements 
were described in depth in syllabi for both sections; verbal class discussion of the reports in the 
traditional classroom was minimal, with students being referred to the syllabus for additional 
information. Approximately equal numbers of students from both classes required additional 
explanation of the report requirements— primarily via e-mail and telephone. 

Results 

Table 3 shows means, standard deviations, and cell sizes for the student scores on the common 
quizzes, online discussions, and final reports for both the traditional classroom and the online 
courses. The much larger standard deviations shown for all the assessment tools for the 
traditional classroom are likely due to the scores of "zero" received by some students for the 
threaded discussions and quizzes— they apparently "forgot" or chose not to access the online 
quizzes and discussions prior to the deadline. In one instance, a student from the traditional 
classroom completed the course, but completed only one of the common assignments used in this 
study. This student's scores were removed from the analysis with no substantial change to the 
study's results. Except for this one case, the zero scores students received are random across 
various students, with no discernable pattern of when they appear; hence, these scores were not 
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removed as outliers because they were considered to represent the performance of the traditional 
classroom course as a whole. Table 4 is a correlation matrix that includes all continuous (ratio) 
dependent variables used in the study. 


TABLE 3: Assessment Method Means by Class 


Type of 
class 


Online 
quiz for 
chapter 
16 
(15 
points 
possible) 

Online 
quiz for 
chapter 
11 
(15 
points 
possible) 

Online 

discussion 

for 

chapter 5 
(20 points 
possible) 

Online 

discussion 

for 

chapter 7 
(20 points 
possible) 

Online 

discussion 

for 

chapter 

11 

(20 points 
possible) 

Written 

report 

(100 

points 

possible) 

Overall 

GPA 

(4.00 

possible) 

Traditional 

Mean 

10.38 

9.86 

15.81 

17.08 

16.30 

90.95 

2.726 

classroom 

N 

37 

37 

37 

37 

37 

37 

37 

course 

Std. 

4.016 

4.547 

5.792 

6.175 

6.476 

17.010 

.508 


Dev. 








Online 

Mean 

11.65 

13.65 

18.81 

19.04 

18.77 

94.19 

3.171 

course 

N 

26 

26 

26 

26 

26 

26 

26 


Std. 

1.056 

1.263 

1.550 

1.371 

4.072 

6.099 

.368 


Dev. 








Totals 

Mean 

10.90 

11.43 

17.05 

17.89 

17.32 

92.29 

2.910 

(combined) 

N 

63 

63 

63 

63 

63 

63 

63 


Std. 

3.196 

4.023 

4.760 

4.883 

5.705 

13.623 

.503 


Dev. 
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TABLE 4: Correlation Matrix of Assessment Instruments 



Online 
quiz for 
chapter 16 

Online 
quiz for 
chapter 
11 

Online 
threaded 
discussion for 
chapter 5 

Online 
threaded 
discussion 
for chapter 
7 

Online 
threaded 
discussion 
for chapter 
11 

Written 

report 

Online quiz 
for chapter 16 

1 






Online quiz 
for chapter 1 1 

.454** 

1 





Online 
threaded 
discussion for 
chapter 5 

.305* 

.136 

1 




Online 
threaded 
discussion for 
chapter 7 

.287* 

.285* 

.475** 

1 



Online 
threaded 
discussion for 
chapter 1 1 

.420** 

.548** 

365** 

.220 

1 


Written report 

.415** 

.368** 

.068 

.534** 

.358** 

1 

Overall GPA 

.317* 

505** 

425** 

356** 

343** 

.271* 


* = correlations sig. at .05 level 
** = correlations sig. at .01 level 
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TABLE 5: Univariate Results with Class (only) as Independent Factor 


Source 

Sum of 
Sqs. 

df 

F 

Sig. 

Online quiz for chapter 16 

24.841 

1 

2.490 

.120 

Online quiz for chapter 1 1 

219.220 

1 

17.052 

.000 

Online threaded discussion 
for chapter 5 

137.143 

1 

6.599 

.013 

Online threaded discussion 
for chapter 7 

58.504 

1 

2.514 

.118 

Online threaded discussion 
for chapter 1 1 

93.306 

1 

2.958 

.091 

Written report 

160.927 

1 

.865 

.356 


The findings were initially surprising; it was expected that students in the traditional classroom 
course would perform better on the quizzes than students in the online course. Often professors 
emphasize material that will be tested, and students have the opportunity to ask questions and to 
observe the professor's personal style including verbiage from classroom lectures often used in 
test question development. One possible explanation for the higher performance of the online 
class is that these students were more experienced in taking quizzes electronically, but this only 
explains the observed disparity for the two same-format quizzes. Another possible explanation is 
that students in the online class were, on average, better students by mere chance. That does not 
explain, however, why the online students’ performances were superior. Their presumed “lower 
quality” pedagogy, with considerably less faculty interaction, should have systematically resulted 
in inferior online performances as they proceeded through their mostly online programs. 

It was also initially presumed that the traditional classroom students would exhibit poorer 
performance in their online threaded discussions when compared to the online course students. 
However, the results showed that the only threaded discussion that was significantly different 
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between classes was the first threaded discussion the traditional classroom students participated 
in. An identical rubric for grading the threaded discussions was included in both courses' syllabi, 
so students knew exactly how their participation would be graded. It is likely that the traditional 
classroom students weren't as experienced in the threaded discussion format; but after the first 
discussion, they were participating at a level equal to the online course. 

For the students' final reports, it was anticipated there would be no difference between the two 
course formats because both the traditional and online students had extensive experience through 
other courses in report writing. All the students were required to turn in a report showing in- 
depth research on a single topic from the class material (their choice of topic). As anticipated, 
univariate testing showed no difference between the two courses. 

During the initial analysis, the effect of individual differences on scoring outcomes was not 
considered. To test the effects of individual differences, a second multivariate test was run, 
adding demographic variables discussed in the method section for which there were significant 
differences between the two classes as covariates. These included grade point average, age, 
number of credits taken during the same term, and whether or not the student was a business 
major. When the demographic variables were added to the model as covariates, the class variable 
(traditional classroom course versus online course) was no longer significant (Wilks' 
Lambda=.792, F=2.01, p> .05), and the only demographic variable that was significant was 
grade point average (GPA) (Wilks' Lambda=.702, F=3.26, p<.05). The univariate results 
revealed that student GPAs influenced two of the six dependent variables: online quiz 1 1 and 
online discussion 5 (see Tables 6 and 7). These results indicated that individual student 
differences may account for any differences between the two classes. When the GPA effect was 
controlled, the course delivery method did not seem to affect student outcomes. Thus, 
equivalency theory is supported by the results - or is it? 
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TABLE 6: Univariate results with class as independent factor and demographic variables 
as covariates 

Results related to class 


Source 

Sum of Sqs. 

df 

F 

Sig. 

Online quiz for chapter 16 

9.656 

1 

.958 

.332 

Online quiz for chapter 1 1 

96.161 

1 

7.650 

.008 

Online threaded discussion for 





chapter 5 

50.679 

1 

2.436 

.125 

Online threaded discussion for 





chapter 7 

16.975 

1 

.691 

.410 

Online threaded discussion for 





chapter 1 1 

106.438 

1 

3.329 

.074 

Written report 

13.690 

1 

.068 

.796 


TABLE 7: Univariate results with class as independent factor and demographic variables 
as covariates 

Results related to GPA 


Source 

Sum of Sqs. 

df 

F 

Sig. 

Online quiz for chapter 16 

39.152 

1 

3.884 

.054 

Online quiz for chapter 1 1 

110.114 

1 

8.760 

.005 

Online threaded discussion for 





chapter 5 

130.668 

1 

6.280 

.015 

Online threaded discussion for 





chapter 7 

20.530 

1 

.836 

.365 

Online threaded discussion for 





chapter 1 1 

121.117 

1 

3.788 

.057 

Written report 

91.415 

1 

.452 

.504 
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Discussion and Conclusion 


At a time when many institutions are turning to online delivery of courses, it is important to 
know if students are in fact gaining knowledge equivalent to traditional classrooms through 
online medium. According to Simonson and colleagues (1999), one of the key elements in 
equivalency theory is "equivalency," or the design of learning environments that provides equal 
experiences for all students, regardless of delivery medium. This research suggests that there are 
no apparent differences between traditional undergraduate classrooms and online education, at 
least after GPA is controlled, thereby supporting equivalency theory. Students in the traditional 
classroom appear to be conditioned to hearing information from the professor and need this 
audio delivery prior to taking quizzes. When this accustomed format is followed prior to taking 
an online quiz, the students perform equivalent to the online students, but in the absence of a 
typical lecture, the traditional students have more difficulty with an online quiz. When overall 
college GPA is taken into account, both teaching modes also appear to provide students with an 
equal basis for performance on the online threaded discussions and written reports. 

But why does GPA need to be controlled for in the first place in order to arrive at support of 
equivalency theory? Why did the students, on average, perform better while in online programs? 
Is it possible that, in a way similar to Chandler’s (1962) explanation of the superiority of the 
divisional form of (large) organizations, that the participants involved in online coursework feel 
more of a sense of responsibility for their work and learning? 

Additionally, evolutionary dynamics may be present. It may be that only diligent, conscientious 
students are able to survive through an online program, while others self-select out of that 
environment, as did one of the initially enrolled online students involved in this study. Online- 
program students who survive the more responsible format may develop better self-directed 
study and learning habits than traditional students. Thus, while equivalency theory may hold for 
the class-by-class level of analysis, there may be benefits to online learning at the program level 
of analysis. Students surviving online programs may in fact have obtained a more effective 
education overall than traditionally educated students. 
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This increase in learning effectiveness, however, raises the question of the efficiency of online 
learning. This format requires far more preparation and input from faculty than the traditional 
classroom setting. Is the increased learning effectiveness worth the tremendous increase in effort 
put forth by a department’s faculty? A future study should analyze the costs and returns of each. 
Perhaps, as suggested by Arbaugh and Duray (2002), it is time for the online course to be priced 
higher than the equivalent traditional course. 

An additional matter for further investigation concerns the results for the Chapter 1 1 quiz. Online 
students performed significantly better than traditional students in this material relative to all 
other course material, suggesting that some types of online content might be particularly more 
effective if offered only online. This leads to the idea of semi-online classes with a mix of 
traditional and online format, depending on the content of the learning material. For example, 
entire encyclopedic-type chapters could be relegated to an online format, thus giving traditional- 
format teachers more time to teach more engaging topics. Perhaps schools will someday offer 
courses consisting of a mixed format of traditional and online components, arranged according to 
the empirical effectiveness identified by studies subsequent to this one. 

Although the small sample size in this study requires caution in generalizing the findings to a 
larger population, it appears that both the traditional classroom and the online teaching methods 
in these classes provide students with an equally effective learning method. Any performance 
differences on the measurements in this study can be explained by the differences in GPA of the 
students taking the two types of classes. Perhaps the students who are drawn to online courses 
are actually better self-learners, intrinsically motivated to acquire knowledge beyond that 
necessary to merely pass the course. Many of the students in the online course were self-reported 
non-traditional students, which may have accounted for the higher overall GPA. This factor, 
however, leaves open the possibility that the online format itself may be responsible for higher 
student GPAs. Therefore, future studies should address the performance of non-traditional 
students and whether the online courses are their better choice for learning. 

Future studies could also proceed in the direction of more experimental control when testing the 
support of equivalency theory as it applies to online versus traditional learning among 


The Journal of Educators Online, Volume 5, Number 1, January 2008 


15 


undergraduate university students. More experimental control has the benefit of bringing to light 
various nuances and limits to the application and relevance of equivalency theory, as this study 
has attempted to demonstrate. Further studies in the direction of greater experimental control 
could include a randomized design from the same pool of students, perhaps tracing the same 
students over a number of courses. The surprising increase in variance among traditional 
classroom student scores (with the observed zeros for some online discussions) also suggests that 
more observations may provide more power to the test of equivalency theory. 

Finally, the nature and testability of equivalency theory itself must be discussed. The field needs 
a theory that points to subtle differences to be supported by carefully designed empirical 
experiments rather than the current state of the field that supports equivalence with the aid of 
general ad-hoc trend studies that cannot reject equivalence. By adding to an emerging body of 
research in online vs. classroom business education that is moving away from designs that only 
compare end of course outcomes toward designs that examine the entirety of courses (Kock, 
Verville, & Garza, 2007; Sautter, 2007) this study contributes to the “second generation” 
comparison literature by moving the field toward the development of just such types of middle- 
range theories. Additional comprehensive comparison studies will allow for a rational scientific 
approach toward the integration of online learning with traditional learning. In further study, we 
may find that the inherent limitations of one format will be mitigated by the advantages of the 
other as they relate to the questions of learning efficiency and effectiveness. 
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